Skip to content

docs(permissions): generate system-context.mdx's declared counts instead of hand-typing them - #17043

Merged
os-trump merged 2 commits into
mainfrom
claude/issue-16919-generate-system-context-counts
Sep 9, 2026
Merged

docs(permissions): generate system-context.mdx's declared counts instead of hand-typing them#17043
os-trump merged 2 commits into
mainfrom
claude/issue-16919-generate-system-context-counts

Conversation

@os-trump

@os-trump os-trump commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Fixes #16919

What changed

check-system-context-census.mjs --fix (aliased pnpm gen:system-context-census) now regenerates every DECLARED_COUNTS sentence in content/docs/permissions/system-context.mdx from a fresh census, instead of only comparing them. It reuses the exact pattern/value pair the COUNTS check already runs — as a write instead of a comparison — so there is exactly one definition of what each number means, read twice. Anchors are untouched: they still have nothing mechanical to repair (symbol anchors encode no position), and --fix refuses loudly, writing nothing, if a sentence no longer parses.

This is direction 1 from #16919, as dispatched: the seven-plus hand-typed count sentences become generated output instead of a human re-deriving and re-typing a number after every CI red.

  • scripts/check-system-context-census.mjs — new regenerateDeclaredCounts() (pure: text in, text + rewrites + errors out), wired into --fix; reportNoFix narrowed to reportNoAnchorFix since counts are no longer in that "nothing to rewrite" bucket; self-test battery renamed and extended (2 → 8 cases) to pin the new write behavior and that anchors still don't get one.
  • scripts/regen-artifacts.mjs — the system-context.mdx row's comment corrected: it used to assert --fix refuses on any declared-count mismatch; it now regenerates them.
  • content/docs/permissions/system-context.mdx — one paragraph in "Maintaining this table" documenting the new generator and telling the next author never to hand-retype a digit.

Why (the guarantee this buys)

The failure #16919 was filed over is silent at every layer that could see it: the merge driver resolves text, and two branches independently (and correctly, for their own tree) writing the same digit into the same sentence merge clean with no conflict — the merged total is neither side's number. Only an independent recomputation (the COUNTS check) ever saw it, and until now the repair was "a human re-derives the number and retypes it," which is the same manual-arithmetic step that produced the original defect.

After this change the repair is pnpm gen:system-context-census, on the tree you actually want counted — mechanical, and it refuses rather than writing a partial page if the wording it reads has moved. It also makes the next hand-edit of one of these sentences loud rather than silent going forward: PR #16860 (open, unrelated) still hand-corrects these same seven sentences to a literal digit on its own branch; when the two branches meet, git now sees a real edit to the sentence structure on one side and a bare digit bump on the other, which is a genuine conflict — not the "both sides typed the same digit" shape that produced this card. That said, I'm not claiming the deeper property that two generator runs against different trees can never coincidentally produce identical text — they still can, for the same reason two independent correct arithmetic results can agree. What's removed is the hand-typing step, which is the actual defect this card names.

--fix still cannot add or drop a table row — a site that arrived or vanished is a human's editorial call, same as before; it only keeps the aggregate counts in lockstep with whatever rows exist.

The counter is live — reading recorded, not re-bumped

Per the dispatch's warning, I checked whether origin/main moved the true count while this branch was open:

  • Branch base (555a89cbd): check-system-context-census: OK — 107 elevation read sites in 20 packages across 45 files...
  • origin/main tip at time of the check (2e8e1185b, in a disposable worktree, no shared state with this branch): same reading, 107 sites, and git diff between the two commits touches none of content/docs/permissions/system-context.mdx, scripts/check-system-context-census.mjs, or scripts/regen-artifacts.mjs.
  • After merging that origin/main into this branch (clean merge, no conflicts — touched files: packages/lint, packages/objectql, packages/rest, packages/spec/scripts/liveness, two changesets; none overlapping this PR's files): re-ran both --self-test and the plain check on head 3139e826a — still 107, exit 0.

So the counter did not move during this round; nothing was re-bumped.

Verification

Ablation — predicted before running, then measured

Predicted: mutating the headline sentence's 107 to 999 on disk makes the gate refuse, naming the census's real value; restoring returns the file to its exact original bytes.

HEAD_BLOB (content/docs/permissions/system-context.mdx) = 267ec7b304f7d2904e871658267556f9b39cf216

anchor 'read at **107' : 1 -> 0     (predicted 1->0, measured 1->0)
anchor 'read at **999' : 0 -> 1     (predicted 0->1, measured 0->1)
MUT_HASH = eb07da9329f0957288ccc109d419c591cbe96ad6   (differs from HEAD_BLOB, mutation confirmed on disk)

$ node scripts/check-system-context-census.mjs
::error::[declared-count] `headline-sites` says 999, the census says 107 (the headline claim in the opening section).
VERDICT command-exit 1   (predicted: exit 1, [declared-count] headline-sites 999 vs 107 — matched)

Restored under trap '... git checkout HEAD -- "$ABS"' EXIT INT TERM. Restoration proof, not exit-code proof:

RESTORED_BLOB = 267ec7b304f7d2904e871658267556f9b39cf216   (== HEAD_BLOB)
git diff HEAD -- content/docs/permissions/system-context.mdx   -> empty

Idempotence, on the same mutation

Predicted: one --fix run repairs the 999 back to 107 exactly (byte-identical to the pre-mutation original); a second --fix run is then a true no-op with the same hash.

MUT_HASH = b58e95c4c7ffa9aa4f5ee4ef63b98c00d240a2c7

--fix run 1: "regenerated 1 declared count(s) from the census: headline-sites 999->107."
  HASH1 = 267ec7b304f7d2904e871658267556f9b39cf216   (== HEAD_BLOB — full repair, not just "a" value)

--fix run 2: "every declared count already matches the census -- nothing to rewrite there."
  HASH2 = 267ec7b304f7d2904e871658267556f9b39cf216   (== HASH1)

HASH1 == HASH2 (idempotent) and HASH1 == HEAD_BLOB (the regenerated sentence is byte-for-byte the original, not merely "a passing value"). Restored again under the same trap; final git status/git diff HEAD clean.

Generated, not edited

Both the ablation and idempotence runs above mutated and repaired the file exclusively through sed/the script's own writeFileSync, never through an editor; the one hand-authored change to this doc in this PR is the new documentation paragraph, made with the file freshly re-read after the ablation script's trap restore overwrote my first attempt at it (caught by re-grepping for the paragraph before proceeding — worth naming since it's exactly the kind of silent loss this card is about, just at the tooling layer instead of the merge layer).

Self-test and check, before and after the origin/main merge

node scripts/check-system-context-census.mjs --self-test   -> all cases passed (both before and after the merge)
node scripts/check-system-context-census.mjs                -> OK — 107 elevation read sites... (both before and after)

Gate reconciliation

node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack against merge-base 2e8e1185b of origin/main and this branch's HEAD (3139e826a) derived 63 commands (36 pnpm, 27 direct node).

  • Run: 63 / 63. NOT-MEASURED: 0. UNRUN: 0.
  • First pass: 59 exit 0 immediately; 4 answered a prerequisite refusal (@objectstack/lint, @objectstack/client-react/@objectstack/client not built — check:doc-formula-expressions, check:doc-security-posture, check:docs-transcript-drift all exit 3 "PREREQUISITE NOT MET", check:skill-examples exit 1 with the same "package is not built" refusal, own dist caveat). None reported as a pass on that first answer.
  • Built pnpm --filter '@objectstack/lint...' build and pnpm --filter '@objectstack/client-react...' build, re-ran all four → exit 0 on all four (check:skill-examples → "258 prose examples type-check across 3 surface(s)"; check:docs-transcript-drift → "4 declared transcript value(s) across 403 page(s) ... equal what the registry derives today").
  • check:route-envelope is in the deriver's Silent bucket (dispatch-gates: a whole-tree-walk gate whose workflow names: lists only its CURRENT members is placed Silent, so it is never derived for the card that adds a new member — measured on check:route-envelope / PR #16730 #16828) and is never derived — run explicitly per the dispatch note: exit 0, "0 ratcheted" everywhere, no new response-writing module (this diff adds none).
  • check:system-context-census — the gate this card is about — is in the derived 63 and was also run explicitly and repeatedly above: exit 0 throughout.
  • check:pm-widening-tells: node scripts/pm/check-widening-tells.mjs --declaration no --diff PR.DIFF (this PR's own diff) → ✓ 3 changed file(s) read, no widening tell on any declared surface.
  • check:pm-governed-merges --test on the final 3-path file list → 0 of 3 path(s) hit the register (5 surfaces) ... NOT governed.

Clause-② stays no as declared at claim time: no new exported symbol, no new key on a published payload, no packages/spec/src/** path.

Direction 2 — sweep for other hand-maintained totals (measurement, not a second fix)

Swept content/docs/**/*.mdx (181 hand-written pages, excluding the auto-generated references/ tree and the release-owned releases/ tree) for a bold or bare integer sitting next to a countable-population noun (sites/packages/files/routes/tests/checks/rows/...), then checked whether any scripts/**/*.mjs references that doc's path literally (the same relationship check-system-context-census.mjs has with its own page).

⭐ Positive control: FIRED. The sweep re-found content/docs/permissions/system-context.mdx itself (4 hits), correctly marked COVERED (scripts/check-system-context-census.mjs, scripts/isystem-census.mjs, scripts/regen-artifacts.mjs, …). Since the detector can re-find the one instance we know exists, its answer about the rest of the corpus is a real reading, not a NOT MEASURED zero.

File Hits Verdict Note
api/error-catalog.mdx 3 COVERED HTTP status codes in a table a gate reads
api/plugin-endpoints.mdx 1 false positive **404** HTTP status, not a population total
automation/approvals.mdx 1 false positive **403** HTTP status, not a population total
automation/flows.mdx 2 COVERED **403** HTTP status near "route", gated incidentally
permissions/system-context.mdx 4 COVERED (positive control) this PR's subject
permissions/tenant-audit-census.mdx 2 COVERED own census script
protocol/backward-compatibility.mdx 1 CANDIDATE — real, filed "all 69 packages" — .changeset/config.json's fixed group and the actual non-private packages/**/package.json count both measure 70 today. Filed as #17039
protocol/kernel/error-handling.mdx 4 COVERED HTTP status codes
protocol/objectui/concept.mdx 1 COVERED HTTP status code

Answer to the card's question: on this sweep, system-context.mdx is not the lucky one — it's one of at least two. The other files that matched are either already covered by a gate (mostly HTTP status codes sitting near a word the regex read as a population noun) or genuinely stale (backward-compatibility.mdx, filed). Not widened into this PR per the dispatch — filed separately, reported below.

This heuristic is a text-shape sweep, not an AST census: it does not claim completeness over every possible phrasing of a hand-maintained total, only over the "bold/bare integer beside a population noun" shape it was built to catch — the same shape this card's own incident had.

Changeset

skip-changeset (label applied on this PR). Nothing published: the diff touches scripts/** (root dev tooling, not part of any published package's src) and content/docs/permissions/** (a docs page, not an npm-published artifact). Per check-empty-changeset.mjs's own header — "this PR edits a CI-internal script... is the textbook skip-changeset case — such a PR releases nothing" — and an empty-frontmatter changeset is mechanically rejected by that gate when newly added, so a changeset file isn't the right carrier here either way. The load-bearing guarantee (generation over hand-typing, and what that does and doesn't buy against future merges) is stated above instead.

Out-of-scope findings


🤖 Generated with Claude Code

https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37


Generated by Claude Code

…hand-typing them

`check-system-context-census.mjs --fix` regenerates every DECLARED_COUNTS
sentence (the headline, the decomposition table, the plugin-sharing share,
the ruling quote) straight from a fresh census, reusing the exact
pattern/value pair the check already runs -- as a write instead of a
comparison. Anchors are unaffected: they still have nothing mechanical to
repair (symbol anchors encode no position), so `--fix` refuses loudly on a
sentence it can no longer parse rather than writing a partial page.

This removes the failure class rather than detecting it one gate at a time:
two branches independently (and correctly, for their own tree) bumping the
same sentence to the same number text-merge clean with no conflict, and the
merged total is neither side's -- which is the failure mode this change
responds to. The fix is to stop hand-retyping the number at all.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zTkyNHJ7TkuN2oXtP5x37
@os-trump os-trump added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 9, 2026 — with Claude
@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation labels Sep 9, 2026
@os-trump
os-trump marked this pull request as ready for review September 9, 2026 04:43
@os-trump
os-trump enabled auto-merge September 9, 2026 04:43
@os-trump
os-trump added this pull request to the merge queue Sep 9, 2026
Merged via the queue into main with commit 2cd4c54 Sep 9, 2026
37 checks passed
@os-trump
os-trump deleted the claude/issue-16919-generate-system-context-counts branch September 9, 2026 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants